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Introduction 

In this paper I present a number of results on cycles on the moduli space A g of 
principally polarized abelian varieties of dimension g. The results on the tautological 
ring are my own work, the results on the torsion of X g and on the cycle classes of the 
■ Ekedahl-Oort stratification are joint work with Torsten Ekedahl and some of the results 
on curves are joint work with Carel Faber. Our results include: 

• A description of the tautological subring - generated by the Chern classes A; of the 
Hodge bundle E - of the Chow ring of A g and its compactifications. 



• As a corollary we find the Hirzebruch-Mumford proportionality theorem for A g and 
bounds for the dimension of complete subvarieties of A g . 

• A bound for the order of the torsion of the top Chern class X g of the Hodge bundle. 

• A description of the Ekedahl-Oort stratification of A g <S> F p in terms of degeneracy 
loci of a map between flag bundles. 

• The description of the Chow classes of the strata of this stratification. This includes 
as special cases formulas for the classes of loci like p-rank < / locus or a-number 
> a locus. 

• The irreducibility of the locus T a of abelian varieties of a-number > a for a < g. 

• A computation of this stratification for hyperelliptic curves of 2-rank in charac- 
teristic 2. 

• A formula for the class of the supersingular locus for low genera. 

It is a pleasure for me to acknowledge pleasant cooperation with and help from 
Torsten Ekedahl and Carel Faber and useful discussions with Helene Esnault, Frans 
Oort and Piotr Pragacz. I also would like to thank Kenji Ueno for inviting me to Kyoto 
where I found the time to write this paper. 

§1. The Tautological Subring of A g and of A g . 

Let A g lTL denote the moduli stack of principally polarized abelian varieties of dimen- 
sion g. This is an irreducible algebraic stack of relative dimension g(g + l)/2. All the 
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fibres A g ®¥ p and A g ®Q are irreducible. This stack carries a locally free sheaf E of rank 
g ("the Hodge bundle") defined by giving for every morphism S — > A g a locally free 
sheaf of rank g which is compatible with pull backs. It is defined as s*Qa/Si where A/S 
is the principally polarized abelian variety corresponding to S — > A g and s is the zero 
section of A/S. If 7r : X g — > „4 3 is the universal abelian variety we have ft* /a = 7r*(E). 

Let be a smooth toroidal compactification of A g . The Hodge bundle can be 
extended to a locally free sheaf on A g . 

The Chern classes Ai of the Hodge bundle E are defined over Z and give rise to 
classes Aj in CH*(A g ), and in CH*(A g ). They generate subrings (Q-subalgebras) of 
CHq(A 9 ) and of CHq(A g ) which we shall call the tautological subrings. 

We shall first describe these tautological subrings. It will turn out that the tauto- 
logical subring of CHq(A 9 ) is isomorphic to the cohomology ring R g -i of the compact 
dual of the Siegel upper half space of degree g — 1, while the tautological subring of 
CH^Ag) is isomorphic to R g . This cohomology ring is of the form 

Rg = Q[ Ul , ...,Ug}/((l + Ul + ... + U g )(l- Ul +U 2 - ...+ (-1)%) - 1) 

and is a Gorenstein ring. As a corollary of this we find the Proportionality Principle of 
Hirzebruch and Mumford for A g and its compactifications. 

We have the following relation for the Chern classes A^ on A g . 

(1.1) Theorem. The Chern classes Xi in CHq(A g ) satisfy the relation 

(1 + Ai + . . . + A 5 )(l - Ai + . . . + (-1)%) = 1. (1) 

The idea of the proof is to apply the Grothendieck-Riemann-Roch theorem to the 
theta divisor on the universal abelian variety X g over A g . We choose this divisor (on 
a level cover) so that its restriction s*(0), with s the zero section, is trivial on A g and 
apply Grothendieck-Riemann-Roch to the line bundle L = O(0): 

ch(^L)=n Sf (ch(L).Td(n 1 Xg/Ag r) 

= 7T*{ch{L) -Td(7T*(E V ))) 

= 7T* (ch(L) ) -Td(E v ) 

by the projection formula. Since R l 7T*(L) = for i > it follows that 7n(L) is a vector 
bundle and it is of rank one since G is a principal polarization. We write ci(7n(L)) = 9 
and find 

fc=0 k=0 vy ' 

Comparison of the term of degree 1 gives 

e = -X 1 /2 + n^+ 1 )/(g + l)l. 

Replace now L by L® n . The term of degree k in ^(J^ Q 9+h /(g + k)\) changes by a 
factor n3+ k . But 7r.(L n ) is a numerical function of degree < n 9 , cf. the arguments of 
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Chai-Faltings on p. 26 of [F-C]. Or, alternatively, using the Heisenberg group we find 



on a suitable cover of A g 



where A is the standard irreducible representation of dimension n 9 of the Heisenberg 
group. Therefore, up to torsion we have 

ch(n,L n ) = n 9 ch(n,L). 

This then proves by induction that 

oo rsg+k 

-*GC^TW ) = lGOTa(A) ' (3) 

In particular we get 

29 = -Ai (the "key formula"). 

Therefore, if we write Xj as the jth symmetric function of ai,...,a g and if we use 
Td(E v ) = Yl( ai /(e ai - 1)) the identity (2) becomes 

9 e a,/2_ e -ai/2 
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and implies Td(E©E v ) = 1. This is equivalent with ch 2 fc(E) = for k > 1 and with (1). 
□ 

(1.2) Proposition. In CHq(A 9 ) we have X g = 0. 

Proof. We apply GRR to the structure sheaf Ox of the universal abelian variety X over 
the stack A g . We get 

chfaOx) = n*(ch(O x ) -Tdi&y) 

= 7T*(l)Td(E V ), 

which gives 

ch(l - E v + A 2 E V - . . . + (-1) 9 A 9 E v ) = 7r,(l)Td(E) v = 0. 
Now for a vector bundle B of rank r we have in general the relation (see [B-S]) 

r 

J2(-V J ch(A J B v ) =c r (B)Td(B)- 1 . 
This gives: X g = in CH*(A g ). □ 

The ring R g is the quotient of the graded ring Q[wi, . . . , u g ] with generators Ui of degree 
% by the relation 

(1 + ux + u 2 + ... + u g )(l- u x + u 2 - ... + (-l) 9 u g ) -1 = 0. 
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This relation implies (by induction) 

UgUg-x ■ ■ -Uk+iul = for k = l,...,g. (4) 
This ring has additive generators 

u^ul 2 ...u\ a with ej-e{0,l}. 

Obviously, we have R g /(u g ) = R g -\. 

It follows that the tautological subring of C£/q(.4. s ) is a homomorphic image of the 
ring R g /{u g ) = R g -i. 

Now consider the moduli space A g ®F p . It contains the loci Vf = Vf(p) of abelian 
varieties with p-rank < /. Their closures in A* g and in A g ®F P define loci again denoted 
Vf. These loci have been studied by Oort and Norman (cf. [01 , N-0]). 

(1.3) Lemma. The subvariety Vf is complete in the moduli space A ( g f) c A g <g> F p of 
rank< f degenerations; in particular Vq is complete in A g . 

(1.4) Corollary. We have X^ 9 ~ 1 ^ 2+f ^ on A g f) . 

Proof. Observe that det(E) is an ample line bundle (its sections are modular forms) 
and so Ai is ample on A*, see [M-B]. On a complete variety of dimension d the <i-th 
power of an ample divisor is non-zero. □ 

(1.5) Theorem. The tautological subring of CHq(A g ) generated by the Xi is isomor- 
phic to Rg-l- 

Proof. By the relation (1 + Ai + . . . + A 5 )(l — Ai + . . . + (—l) 9 X g ) = 1 we get a quotient 
ring of R g . Since moreover X g = we get a quotient of R g /(u g ) = R g -\. This ring 
R g -i is Gorenstein and its top degree elements are proportional. We now use the fact 
that A g <8>F p has a complete subvariety of codimension g, namely the p-rank zero locus. 
The existence of of a complete subvariety of dimension g{g — l)/2 in A g <E> F p for every 

prime p and the ampleness of Ai on A g <8> F p imply Xf^ g 1 7^ 0. This implies that in 
the quotient of R g -i the one-dimensional socle does not map to zero, hence that the 
quotient is isomorphic to R g -\. Or more explicitly, consider the set of 2 S_1 generators 
of the form 

X T X 2 ■ ■ ■ K-i with ^ e {0, 1}. 
Order these elements A e with e G {0, lexicographically. Suppose we have a relation 

$> £ A £ = 

in CHq(A g ). Suppose that e' is the 'smallest' exponent. Let e" be the complementary 
exponent with e' + e" = (1, . . . , 1). Then we have 

A e »(^a e A e ) = <vA 3 _i • • • Ai = 0, 

and this implies that a e > = 0. By induction the theorem follows. □ 
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(1.6) Corollary. In CH^(A g ) we have: A? (s ~ 1)/2 ^ and \\+^9-i)/2 = Q 

Proof. The first statement was given in (1.4). In the ring R g -i all top monomials (of 
degree g(g — l)/2) are proportional, hence Af^ -1 ^ 2 is a non-zero multiple of the top 
monomial A g _iA g _ 2 • • • Ai, so 

x g( 9 -i)/2+i 

is a non-zero multiple of A g _iA 3 _2 • • • A2A1, 

which is zero by (4). □ 

(1.7) Corollary. Let F be a Geld. A complete subvariety of A g ®¥ has codimension > g. 

Proof. If Z is a complete subvariety of dimension m then A™ ^ on Z. Since g(g — l)/2 
is the highest power of Ai which is not zero on A g the result follows. □ 

For a discussion of questions concerning complete subvarieties of A g we refer to 

[02]. 

Now we come to the tautological ring of a suitable toroidal compactification of A g . 
We shall consider various compactifications of A g . We choose a suitable compactification 
A g as constructed in [F-C]. We let A* be the minimal ('Satake') compactification as in 

[F-C]. Furthermore, we let A g ^ be the moduli space of rank 1-degenerations, i.e. the 
inverse image of A g U A g -i C A* under the natural map q : A g — > A*. This space 

Ag = A g 1 ^ does not depend on a choice A g of compactification of A g . See [M3]. 

Let Q be the 'universal' semi-abelian variety over A g with zero section s : A g — > Q. 
Then we have E = s*(LieQ) y . Moreover, let D = A g — A g be the divisor at infinity. 
Then we have the isomorphism (cf. [F-C]. p. 117): 

Sym 2 (E) = n\logD). 

Since the class X g vanishes on A g it can be represented in the form X g = i*(x) with 
x a class in Ci^Q _1 (L>) and i: D — > A g the embedding of the boundary. By applying 

Grothendieck-Riemann-Roch to the structure sheaf on the semi-abelian variety over A g 
(as in (1.2)) one gets an expression for x in CHq(D), see [EFG]. 

As we shall see later in characteristic p a multiple of the class X g is represented in 
the Chow ring by the class of a complete subvariety Vq of A g . More generally, a multiple 

of X g -f is represented by a complete subvariety of A g , the moduli space of rank < / 
degenerations. 

(1.8) Lemma. The class q*(X g ) in CHq(A*) is represented by a multiple of the fun- 
damental class of the boundary B* = A* — A g . 

Indeed, X g is zero on A g ; for dimension reasons q*X g is represented by a multiple 
of the fundamental class of B*. 
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(1.9) Proposition. The cycle class [B*] of the boundary is the same in the Chow 
group CHq(A*) as a multiple of the class of the image of A g -i in A g under the map 
[X] i— > [X x E] , with E a generic elliptic curve. 

Proof. Consider (for g > 2) the space A g -i^i ~ ^ g -i x <A\ in A g . Since A\ is the afhne 
j-line we find a rational equivalence between the cycle class of a generic fibre A g -\ x {j} 
and a multiple of the fundamental class of the boundary B*. □ 

Let B g be the cycle on Ag defined by the semi-abelian varieties which are trivial 
extensions 

1 — > G m — > G — > — > 0, 

where X g _i is a (g — l)-dimensional abelian variety. The cycle B g ~ «4 3 -i is of codi- 
mension (7 in Ag 1 ^ and extends to A g . 

The following proposition describes the class of B g in cohomology. 

(1.10) Proposition. The cohomology class of B g and of A g -i x {j} are both equal 
to (—l) 9 X g /C(l — 2g). ( Here ((s) denotes the Riemann zeta function.) 

Proof. The intersection of the closure of A g -\,\ with the boundary divisor in A g is 
the closure of B g . Using the rational equivalence of B g and A g -\ x {j} for generic 
j and the fact that the class of A g -i x {j} on A* was a multiple of q*\ g the result 
follows by pull back. To find the multiple we look at characteristic zero and integrate 
the Sp(2g, M)-invariant forms representing the Aj. □ 

Examples. We have in cohomology: 
g=l [5 1 ] = 12A 1 ; 
9 = 2 [B 2 ] = 120A 2 . 

Consider the moduli space A' = A g of rank < 1 degenerations. Let D° be the 
closed subset corresponding to rank 1 degenerations. The divisor D° has a morphism 
to <fi : D° — > A g -i which exhibits D° as the universal abelian variety over A g -\. The 
fibre over x G A g -i is the dual X g -\ of the abelian variety X corresponding to x. 
The 'universal' semi-abelian variety G over X g -i is the G m -bundle obtained from the 
Poincare bundle P — > x X g -i by deleting the zero-section. We have the maps 

G = P - {(0)} - x X g _ x ^ 

The divisor D° contains the subvariety B g corresponding to the trivial extensions 

1 — > G m — > G — > — > 

and this is a codimension (7 cycle S g ~ ^3-1 i n A' g . 

Consider now the cotangent bundle to G at the zero section t of G — > -X^-i- We 
have an exact sequence 

0^?*E 5 _i^E 9|DO ^L/^0, 
6 



with U a pull back of a line bundle on A* 1 . Now U is trivial since the restriction of 
Kg to B g is a direct sum of E g _i and a trivial line bundle. 

Consider X', the compactified family of semi-abelian varieties over the moduli stack 
A' g of rank < 1 degenerations. A semi-abelian variety which is a G m -bundle over an 
abelian variety X g _i is compactified by taking the P 1 -bundle associated to the G m - 
bundle and then identifying the 0- and the oo-section under a shift £ G X g -\. The 
image of the zero section of the P 1 -bundle maps to a codimension 2 cycle A, the locus 
of singular points of the fibres of X' — > A' g . We have 

A = Xg-l XAg-1 Xg-1- 

We analyze O 1 = ftx^/A' g - We have an exact sequence 

-> O 1 -> tt*(E) -> 0, (5) 

Here is a sheaf with support on A, the codimension 2 cycle. Let u be a fibre coordinate 
on a (Gr m bundle over X g -\. A section of 7r*(E) is given locally by du/u. Pull a section 
back to the P 1 -bundle and take the residue along the 0- and oo-section. The residue 
map yields an isomorphism on A' g 

where A is the etale double cover of A corresponding to choosing the branches (0 and oo) . 
(1.11) Main Theorem. The tautological subring of A 9 in CHq(A 9 ) is isomorphic 

tO Rg. 

In order to prove this we shall apply GRR to the G-divisor again. We can do this 
on a level cover of A g for a line bundle L = 0(T) trivialized along the zero section. But 
we start in codimension 1 and therefore we work on A' g . There we have: 

chimXL®* 1 )) = 7r*(e nT • Td v (0 A )- 1 )Td v (E). (6) 
In particular, for n = 1 we have 

ch(7r,{L)) = n*(e T ■ Td v (0 A )" 1 )Td v (E). (7) 

We write 

ch(7r,(L® n )) = 1 + 9{ n) + 9 { 2 n) + . . . 
and set = 9. In equation (7) we compare terms of codimension 1: 

9^ ] = -y ■ ^[e nT ■ Td v (O A )- 1 )] + n*[e nT ■ Td v (OA) _1 )]i 
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(1.12) Lemma. We have 9 = — Ai/2 + 5/8, where 5 is the class of the 'boundary' D. 
Proof. First we do the case g = 1. We have 

1 + 9 = (tt*(T + T 2 /2) + 7r*(A/12))(l - Ai/2). 

Let S be the zero-section. By Kodaira's results on elliptic surfaces we have S 2 = — Ai; 
so the normalized T is T = S + tt*(Ai) with T 2 = S ■ tt*(Ai), i.e. tt*(T 2 /2) = Ai/2. We 
find 

0= -Ai/2 + Ai/2 + <5/12. 

We can rewrite this as 

= -Ai/2 + 5/8- 

For general # we have a priori = — Ai/2 + aS. Restriction to the space of products 
of elliptic curves -4i,...,i gives a = 1/8. □ 

As a corollary we find 

9{ n) = n 9 (-X 1 /2) + (n 9+1 + 2n 9 ~ 1 )5/2A. 

To finish the proof we now work on the whole space A g and we also introduce a (suit- 
able) level t structure and then apply Grothendieck-Riemann-Roch on the moduli space 
-4. g [£]. The Hodge bundle is a pull back, but since the natural maps A g [£] — > A g are 
ramified over the divisor D one can separate the contributions from the 'interior' and 
the boundary in the analogue of (6) by their dependence on I. This leads to the fact 
that the pull back of the formula e _Al / 2 = Td v (E) holds on the spaces «4 9 [^]- We refer 
to [EFG] for the details of the proof. 

A top monomial u a in R g can be written as a multiple m a u\U2 ■ ■ ■ u g with m a G Z 
using the relation (1). Therefore, the degree degA a equals m a deg A1A2 . . . A 5 ([»4 3 ]). 
The ring R g is also the Chow ring (and cohomology ring) of the Lagrangian Grassmann 
variety Y g of maximal isotropic subspaces of a symplectic complex vector space of di- 
mension 2(7, see [P] and the references there. If we identify R g with CHq(Y g ) then we 
find as a corollary: 

(1.13) Theorem. (Proportionality Principle of Hirzebruch-Mumford) The character- 
istic numbers of the Hodge bundle are proportional to those of the tautological bundle 
onY g : 

\ a ([A g ]) = (-l) G ±f[C(l-2k).u«(lY g ]) 

fc=i 

with G = g(g+ l)/2. 

Proof We have A a ([^4 g ]) = c(g) x -u a ([Y g ] for some constant c(g). To evaluate the 
constant one can compare the Euler number of A g (in a suitable sense; actually for 
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fi(log-D) = Sym 2 (E)) and the Euler number of Y g . The Euler number of Y g equals 2 9 . 
We know the Euler number of A g by the work of Siegel and Harder: 



#Wu(R) 2 gl fx 2 

= C(-l)C(-3)...C(-2</ + l). 

This proves the result. □ 

Define a proportionality factor by 

p( 9 )=(-D c n^ 

i=i 

Since degitiw^ ...« 9 = 1 we find deg A1A2 • • • X g = p(g). Similarly, using the structure 
of R g we have 

1 — (-1) G 1 



nLiC(i-2fe) v ; 2 9 

and thus get 

A? = p(g)G\ f[ 



ii(2t-l)ll 

This has to be interpreted with care (in the orbifold sense). 
Some examples. We have 

p(0) = 1 
p(l) = l/24 deg Ai = 1/24, 
p(2) = 1/5760 deg Af = 1/2880, 
p(3) = 1/2903040 deg A? = 1/181440. 

For the classical Proportionality principle of Hirzebruch we refer to his collected works, 
[Hi 1]. In a letter to Atiyah Hirzebruch sketches how one obtains a copy of R g in the 
cohomology of suitable compact quotients of the Siegel upper half space, cf. [Hi 2]. 



Proposition (1.2) shows that the top Chern class X g of E vanishes in the rational 
Chow group CHq(A 9 ). So X g is a torsion class on A g . Mumford proved in [Ml] that 
the order of Ai in CH 1 (Ai) is 12. In [EFG] we shall prove the following bound for the 
order of the torsion class X g . 

(1.14) Definition. Let n g be the greatest common divisor of all p 2g — 1 where p runs 
through all primes greater than 2g + 1. 

We have a little lemma. 
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(1.15) Lemma. We have 

\\ n i = 1| ([ -~j^ ]Dp ( = multiple of denominator of p(g) ). 
i=l p prime ^ 

(1.16) Theorem. On .A 9 we have: (g - 1)1 (Ui=i n i) X g = °- 

Example i) For g = 1 we get 24Ai = which is off by a factor 2. ii) For g = 2 we get 
24 • (16 • 3 • 5)A 2 = 0. hi) For g = 3 we get 2 • 24 • (16 • 3 • 5) • (8 • 9 • 7)A 3 = 0. 

We refer to [EFG] for other cycle relations in the Chow ring of A g . 
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§2. The Cycle Classes of the Ekedahl-Oort Strata. 



Ekedahl and Oort introduced a stratification of the moduli space A g ® F p , cf. Oort's 
paper [03] in this volume. It is defined by analyzing for abelian varieties X the action 
of Frobenius and Verschiebung on the group scheme X\p], the kernel of multiplication 
by p. The strata include the well-known loci Vf of abelian varieties of p-rank < / (for 
< / < g) and the loci T a of abelian varieties with a-number > a (for < a < g). 

We shall describe these strata in a somewhat different way using the Hodge bundle 
E. We then can apply theorems of Porteous type to calculate the cycle classes of these 
loci. Such Porteous type formulas are obtained by applying results of Fulton and of 
Pragacz on degeneracy maps between symplectic bundles. The cycle classes all lie in 
the tautological ring. We get explicit formulas for the loci Vf and T a generalizing 
Deuring's formula for the number of supersingular elliptic curves and the formula for 
the number of superspecial abelian varieties (with a = g). 

In the following we shall study the moduli space A g <8> ¥ p of principally polarized 
abelian varieties in characteristic p. For simplicity we shall write A g instead of A g <&F p . 

Recall the canonical filtration on the de Rham cohomology of a principally polarized 
abelian variety X as defined by Ekedahl and Oort. We write G = H\ R {X) on which we 
have a a-homomorphism F and a a _1 -homomorphism V . For a moment we shall ignore 
the o" ± -linearity. We have FV = VF = 0. The 2gr-dimensional space is provided with a 
symplectic form (, ) and F and V are adjoints: (Vg,g ! ) = (g,Fg f ). For any subspace 
H of G we have (VH) 1 - = F _1 ( J H"- L ). The spaces V(G) = F _1 (0) and F(G) = V _1 (0) 
are maximally isotropic subspaces of dimension g. 

To construct the canonical filtration one starts with C G and constructs finer 
nitrations by adding the images of V and the orthogonal complements of the spaces 
present. This process stops. The filtration obtained is stable under V and under , 
hence under F _1 as well and is called the canonical filtration. The canonical filtration 

c Ci c ... c C r c C r+ i c ... c C 2r 

satisfies C r = V(C2 r ) and C^_ { = C r+ i. This filtration can be refined to a so-called 
final filtration by choosing a V- and -'--stable filtration of length 2g which refines the 
canonical one. In general, there is no unique choice for a final filtration. We thus get a 
filtration 

OcGiC...cG 9 c G g+1 c ... c G 2g 

which satisfies V(G2 g ) = G g , Gg_ t = G g+ i and now also dim(Gj) = i. The associated 
final type is the increasing and surjective map 

z/:{0,1,2,...,2<7}^{0,1,2,...,<7} 

satisfying 

u(2g -i) = v{i) -i + g for < z < g (9) 

obtained by v(i) = dim(V (Gi)) and v(0) = 0. The canonical type is the restriction of v 
to the integers which arise as dimensions of the Cj. Although the final filtration is not 
unique, the final type is. (The dimensions between two steps of the canonical filtration 
either remain constant or grow each step by 1.) 
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Example. Let X be an abelian variety with p-rank / and a(X) = 1 (equivalently, on 
G g the operator V has rank g — 1 and semi-simple rank g — f). Then the canonical type 
is given by the numbers {rank(Ci)} 

= {0<f<f + l<...<g-l<g<g + K...<2g-f-l<2g-f<2g} 

and 

K/) = /, K/ + l) = /, 
I /(/ + 2) = /+l,...,%)=^-l,...,z/(2^-/-l) = ^-l, 

u(2g-f) = g, u(2g)=g. 

It is not difficult to see that there is a bijection between the set of final types and the 
set of canonical types and we have 2 9 of them. Note that in view of (9) the function v 
is determined by its restriction to {1, 2, . . . , g}. This restriction is again denoted by v. 

The Ekedahl-Oort stratification of A g is obtained by looking for each geometric 
point of A g what the canonical type (or final type) is. The set Z v of all abelian varieties 
which have given final type v is locally closed and these Z v define a stratification, cf. 
[03]. 

I prefer to describe the combinatorial datum of a final type v by a partition \i = 
{g > A*i > A*2 > • • • > A*r} as follows: 

fij = #{z : 1 < % < g, U (i) < i - j}; 

equivalently, we can visualize it by the associated Young-type diagram with \ij squares 
in the j-th layer (i.e. by putting a stack of % — v{%) squares in position %): 



12 3 ... g-1 g 

This example corresponds to : i = 1, g} = {1, 2, . . . , g — 5, g — 5, g — 4, g — 4, g — 
3, g — 3} and to fx = {5, 3, 1}. 

(2.1) Definition. We call a partition fi admissible if g > fj,\ > fii > . . . > fi r > 0. We 
call the function v:{l,...,g} — > {1, . . . , g} admissible if 

< v(i) < v{% + 1) < v{%) + 1 < g + 1 
for 1 < z < (7. The number := is called the area of the diagram. 

The notions 'admissible' diagram (or partition) and 'admissible function' v and final 
type are all equivalent. The set of admissible diagrams carries a partial ordering in the 
obvious way: /i > /J,' if \ii > for all i. 

We shall now give another approach to these strata by defining them globally 
on a flag space over A g . Our starting point is the observation that if we define for 
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a final filtration E t (1 < i < g) another filtration by F g = ker(V) = V 1 (0), by 
F g+i = V~ 1 (E i ) for i = 1, g and by F g _i = F^~ +i then we have 

V{Ei) C E v{i) <=^E t C V-\E v{i) ) = F g+V{i) ^ dim(£, n F g+v{i) ) > i. (10) 

Working now globally, we let S be a scheme in characteristic p and let X — > S 1 be 
an abelian variety over S 1 with principal polarization. Then we consider the de Rham 
cohomology sheaf Ti dR {X/S). It is defined as the hyper-direct image 7V-n*{Ox — *• 
^at/s)- ^ * s a l° can y f ree sheaf of rank 2g on S 1 equipped with a non-degenerate 
alternating form (cf. [O]) 

(,y.n dR (x/s)xn dR (x/s)^o s . 

Indeed, the polarization (locally in the etale topology given by a relatively ample line 
bundle on X/S) provides us with a symmetric homomorphism p: X — > X and the 
Poincare bundle defines a perfect pairing between H dR (X/S) and H dR {X/S). Moreover, 
we have an exact sequence of locally free sheaves 

-> tt^O 1 ) -> H\ R (X/S) -> flV-O* -> 0. 

We shall write HI for the sheaf H dR (X g / 'A g ) and # for A^,. We thus have an exact 
sequence 

0^E^H^E v ^0. 

The relative Frobenius F : X — > X^ p > and the Verschiebung V : X^ ^ X satisfy 
F-V = p • id^(p) and V ■ F = p- i&x and they induce maps in cohomology, again denoted 
by F and V: 

F:H (p) ^H and V:U^U (p \ 

Of course, we have FV = and VF = and F and V are adjoints. This implies that 
Im(F) = ker(V) and Im(V) = ker(F) are maximally isotropic subbundles of H and H.^ . 
Moreover, since dF — on Lie(X) it follows that F = on E and thus Im(V) = K^ P K 
Verschiebung thus provides us with a bundle map (again denoted by V): V : H -> E^. 

Consider the space of symplectic flags JF = Flag(H) on the bundle H. This space 
is fibred by the spaces .F^) of partial flags 

E, C E, +1 C...CE S . 

So .FW = Flag(H) and JF^) = A g and there are natural maps 

7r M+1 :^)^^ +1 ). 

The fibres are Grassmann varieties of dimension i. The space .F* is equipped with a 
universal flag. On T the Chern classes of the bundle E decompose into their roots: 

\ = a i (£ 1 ,...,£ g ) with e i = c 1 (E i /E i _ 1 ). 

Given an arbitrary flag of subbundles 

C Ei C . .. C E g = E (11) 
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with rank(Ej) = i we can extend this to a symplectic filtration on EI by putting 

E g + i = (Eg-i)- 1 . 

By base change we can transport this filtration to MS P ' . 

We introduce a second filtration by starting with the isotropic subbundle 

W g = ker(V) = V-\0) C H 

and continuing with 

¥ g+i = V- 1 (E^ ) ) for l<i<g. 

We extend it to a symplectic filtration by setting E g -i = (Fg+i)- 1 . We thus have two 
nitrations E. and F. on HI. 

Example, i) Let X be an ordinary abelian variety. Then Lie(X) = Lie(fi) with \i the 
multiplicative subgroup scheme of X[p] of rank p 9 . It follows that V is invertible on 
Lie(X) v = u(X), i.e. ¥ g fl E g = (0). ii) If X is a superspecial abelian variety (i.e. X 
without its polarization is a product of supersingular elliptic curves) then V = on 
uj(X) so that rk(Ei n ¥ g ) = i. 

These two (extreme) examples show that the respective position of the two nitra- 
tions E. and F. for an abelian variety X gives information on the structure of the kernel 
of multiplication by p on J. These respective positions are encoded by a combinatorial 
datum, e.g. v or to be more precise, by an element of a Weyl group. We shall associate 
strata to such data. 

To either v or \x we now associate an element of the Weyl group of the symplectic 
group. The Weyl group W g of type C g in Cartan's terminology is isomorphic to the 
semi-direct product S g x (Z/2Z) 3 , where S g acts on (Z/2Z) 9 by permuting the g factors. 
Another description of this group is as the subgroup of 5^ of elements which map any 
symmetric 2-element subset of the form {i, 2g + 1 — i} of {1, ... , 2g} to a subset of the 
same type: 

W g = {a E S 2g : o-(i) + a(2g + 1 - i) = 2g + 1 for i = 1, . . . , g}. 
An element in this Weyl group has a length and a codimension: 

£(w) = #{z < j : w(i) > w(j)} + #{z < j : w(i) + w(j) > 2g + 1} 

and 

codim(ty) = #{z < j : w(i) < w(j)} + #{z < j : w(i) + w(j) < 2g + 1}. 
We have the equality 

i{w) + codim(ty) = g 2 . 

To a function v we associate the following element of the Weyl group, a the per- 
mutation of {1,2,..., 2g\ : let 

S = {h, i 2 , • • •} = {1 < i < g : = v{% - 1)} 
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with ii < i 2 < . . . given in increasing order. Let 

s c = {n,j 2 ,...} 

be the elements of {1, 2, . . . , g} not in S, in increasing order. Then one gets the per- 
mutation of S2 g defined by v by writing g + k at position jk for k = 1,2,... and at 
position ik for = 1, 2, — We finish off by putting 2g + 1 — k at position 2g + 1 — i if fe 
is written at position i. We obtain a sequence s which is a permutation of {1,2,..., 2g}. 

Alternatively, using diagrams, we can describe the element w = as follows. 
Let ti be the operator on the set of diagrams which is 'remove the top box of the i-th 
column'. We let the ti act from the right. Given an admissible diagram \x we consider 
the complementary diagram 

v c = {g,g- - fi 

which is also an admissible diagram (partition). We can successively apply operators 
ti to it such that after every step we obtain an admissible diagram and such that after 
steps we obtain the empty diagram (// c ) ■ ti 1 - ■ -ti e = 0. We remove first the top 
layer, then the next one and so on. For 1 < i < g let Sj G S2 g be the permutation 
(i, i + 1) (2(7 — i, 2g + 1 — i) and let s g = (g,g + 1) e S2 g - Now associate to the diagram 
\x the element of the Weyl group: 

/j^w^ = s il - ■ -s ir 

Each diagram thus yields an element in the Weyl group. The admissible partitions yield 
2 9 elements of the 2 9 gl elements of W g . 

Example, g = 3 
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We can associate to the map V : H -»• E( p ) and an element w a degeneracy locus 
U w and in particular to an admissible diagram fi a degeneracy locus = U w ^ in 
T = Flag(H) . Intuitively, U w is defined as the locus of points x such that at x we have 

dim(E, n Fj) > #{a < % : w(a) < j} for all 1 < i, j < g 
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or equivalently, that ker(V) n E, > i — (= the number of squares in the diagram 
/U in position i). For the precise definition we refer to Fulton [F]. Note that for the 
admissible diagrams we do not use the full filtration of E, but only ¥ g . 

We first look at the case of the empty diagram \i = or equivalently, that v = 
{1, 2, 3, ... , g}. The degeneracy conditions say that (cf. (10)) 

V(Ei)CE^ i = l,...,g. 

i.e. we are looking at the space U{$y of symplectic filtrations on E which are compatible 
with the action of V. The codimension of this space in JF is g(g—l)/2, hence dim(U^y) = 
g(g + l)/2 = dim(Ag). We have a finite map tt : U{$y — > A g of degree 

deg(yr) = (1 +p)(l + p + p 2 ) . . . (1 + p + p 2 + . . . + P 9 ' 1 )- 

This space W = U{$y can be seen as a component of the moduli space T (p). Indeed, 
a filtration of the subgroup scheme X\p] corresponds to a filtration on H\ R . Fix g and 
let Tq{p)' be the functor that associates to a scheme £ the set of isomorphism classes 
of principally polarized abelian schemes X over S plus a F-stable symplectic filtration 
on X\p\. 

(2.2) Proposition. The degeneracy cycle U% is the algebraic stack representing the 
functor Folp)'. The stack is fibred by finite morphisms 

with deg(7rj ) j + i) = 1 +p+ . . . +p l . It contains the degeracy loci for all 2 9 partitions 

The stacks come with universal (partial) flags Ej C . . . C E 5 . We denote by 
Xj(i) = Cj(Ej) the Chern class in CHq(U^). We also have tautological quotient bundles 
Li = Ej/Ej_i. We denote by U the Chern class of Lj. We have (Xj)(i) = (Jj(h, • • • , h), 
the j-th elementary symmetric function of the l±, . . . , ij. 

Next we look at the cases where \x is a partition of the form \i = {^i = g — /}. The 
corresponding degeneracy loci classify the loci of p-rank < /. It is well-known by Oort 
(cf. [O-N]) that the codimension of Vf in A g equals g — f. They admit the following 
explicit description. The pullback of V g -i to U$ consists of g components, say Z\, Z g . 
An abelian variety of p-rank g — 1 and a = 1 has a unique subgroup scheme ct v . The 
index % of Zi indicates where for the generic point of Zi the a p can be found: a p C Gj, 
a p <£ Gi-i. 

Then the pull back of Vf consists of 

n~\V f )= J2 Z s , 
#S=g-f 

where for a subset S C {1, . . . , g} the cycle Z s is defined as 
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Then one sees easily that the cycle V g -f on A g is obtained as the push forward of 

z g n . . . n z g _ f+1 . 

(2.3) Lemma. The cycle class of Z t is equal to (p — 

Proof. It is described as the locus where det(V) : Li — * vanishes. By viewing 
det(V) as a section of l\ p ^ <g> L~ x the result follows. □ 
We find after a calculation: 

**([V f }) = (p-l)9-f\ g -f 

on U.%. Using the push forward of the Tii^+i on the classes l{ and extending the result 
to a compactification A g we get: 

(2.4) Theorem. The cycle class of Vf, the p-rank < f locus, in the Chow ring 
CH^(A 9 ), is given by [Vf] = (p - l){p 2 - 1) . . . (p*"/ - 1)A 5 _,. 

(2.5) Corollary. (Deuring Mass Formula) Let g = 1. We have 

^#Aut(£) 24 ' 

where the sum is over the isomorphism classes over ¥ p of supersingular elliptic curves. 

Proof. The formula gives (p — l)Ai and by (1.10) this equals 8/12. The class of 5 is 
equivalent to 1/2 times the class of a 'physical' point of the j-line because the degenerate 
elliptic curve corresponding to 6 has 2 automorphisms. □ 

Another case where we can find an explicit formula is the case of the locus T a . This 
corresponds to the case \i = {a, a — 1, . . . , 2, 1}. But here we can work directly on A g . 
The locus T a on A g may be defined as the locus 

{x G A g : rank(V)| Eg < g - a}. 

We have T a+ i C T a and dim(T 5 ) = 0. 

Pragacz and Ratajski, cf. [P-R], have developed formulas for the degeneracy locus 
for the rank of a self-adjoint bundle map of symplectic bundles globalizing the results 
in isotropic Schubert calculus from [P] . Before we apply their result to our case we have 
to introduce some notation. 

Define for a vectorbundle A with Chern classes the expression 

i 

Qij(A) := ciiCij + 2 ^2(-l) k a i+k aj-k for % > j. 

fc=i 

For a admissible partition (3 = (Pi, (3 r ) (with r even, (3 r may be zero) we set 

Q/3 = Pfaffian(a;ij), 

where the (xy) is an anti-symmetric matrix with = . Applying the Pragacz- 

Ratajski formula to our situation gives the following result: 
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(2.6) Theorem. The class of the reduced locus T a of abelian varieties with a-number 
> a is given by 

j2Q^ {p) )-Q P (a)-p(^n, 

where the sum is over the admissible partitions (3 contained in the partition p(a) = 
(a, a — 1, a — 2, 1). 

Example: 

[Til =p\ 1 - Ai 

[T2] Hp - i)(p 2 + l)(AiA 2 ) - (p s - 1)2A 3 

[T 9 ] =(p-l)(p 2 + l)...(^ + (-l) s )A 1 A 2 ---A 3 . 

As a corollary we find a classical result of Ekedahl (cf. [E]) on the number of principally 
polarized abelian varieties with a = g: 

(2.7) Corollary. We have 

E ^XuW = ( " 1)G2 " fl [ W + ' c( " 1)c( " 3) ' ' ' c(1 " 2g) ' 

where the sum is over the isomorphism classes (over ¥ p ) of principally polarized abelian 
varieties of dimension g with a = g. 

Proof. Combine the formula for T g with the Proportionality Theorem. □ 

For each element w we now find a degeneracy locus U w . In particular we find such 
a locus for each partition \x in T and the W M actually lie in U$. It is known that 
T g is zero-dimensional, cf. (2.7). This implies that the codimension of each equals 
codim(w(//)) = \n\. We can apply a theorem of Fulton to determine the class of the 
degeneracy locus in CH^J 7 ). For an admissible diagram \i = {^i > . . . > \i r > 0} 
Fulton defines a determinant ('Schur function') 

A M (xi) = det(a; Mi+ j_ i )i< i j< r ; 

this is a polynomial with integer coefficients in the commuting variables x±, X2, ■ ■ ■■ We 
define a 'double Schubert function' by putting 

A(x,y) = A gj g_ h ___ jl (a i (x 1 ,...,x g ) +a i (y 1 ,...,y g )). 

The operators di ('divided difference operators') on the ring Z[xi, x 2 , . . . ,] are defined 
by setting for F(x) £ Z[xi, . . . , x g ] 

di(F(x)) = F{ t-^ X)) 
d g (F(x)) = EM^mM iU = g. 

We put 

A = A(x,y) = Ag y g_ ly ,„ A (a i (x 1 ,...,x g ) + a l (y 1 , . . . ,y g )). 



18 



We shall apply the result of Fulton and state an abstract formula for our cycle classes. In 
principle one then can calculate the push forward algorithmically. But it seems difficult 
to get closed formulas for the cycle classes of these push forwards. 

By applying a theorem of Fulton [F] we find: 

(2.8) Theorem. Let \i be an admissible diagram whose corresponding element io M in 
the Weyl group is written as Si e - ■ • . The cycle class of the degeneracy locus in 
CH^(F) is given by 

u, = d u ---d n {( J] (xi- yi )).A(x,y))i {xi=plitVi= _ h} 

i+3<9 

(2.9) Corollary. The push forward of the class of [U^] under it is given by 7r*(w M ) and 
is a multiple of the class of the reduced cycle Z^, the multiple being equal to the number 
of final hltrations refining the canonical filtration associated to ji. The class belongs to 
the tautological ring. 

We can calculate these classes from this formula in an algorithmic way. But is 
seems difficult to get closed formulas in the general case. We list here the formulas for 
(7 = 3. The multiplicity is given by the factor in square brackets. 

Formulas for (7 = 3. 

M{0}) = [(i+p)(i + p + p 2 )] 

7T*({1}) = [(1+P)]X (l-p)Ai 

7r*({2}) = (p-l)(p 2 -l)A 2 
tt*({1, 2}) = [(1 + p)] x { (1 - p) (1 + p 2 ) X 1 X 2 -2 (-1 + p 3 ) A 3 } 
^({3}) = (p - l)(p 2 - l)(p 3 - 1) X 3 
tt*({1, 3}) = [(1 +p)} x (-1 +p) 2 (l-p + p 2 ) Ai A 3 
tt*({2, 3}) = (-1 + pf (1 + p) (-1 + p - p 2 ) (1 + p + p 2 ) A 2 A 3 
tt*({1, 2, 3}) = [(1 + p 3 )] x (p - 1) (p 2 + l)(p 3 - 1) Ai A 2 A 3 . 

The canonical filtration on H\ R (X) is the most economical one with respect to the 
operation V. The types of this filtration correspond to the elements of (Z/2Z) 3 in the 
Weyl group (Z/2Z) 5 x S g . The other types of nitrations are obtained by applying an 
element of S g to the right. Therefore, in the closure of a stratum we find also U w 
with w {'L/2'L) 9 . This explains the phenomenon observed by Oort in [03], 4.6. We 
can use Pieri-type formulas to get results on the boundary of strata. In particular one 
obtains the result of Ekedahl-Oort that the class Ai is torsion on the open strata. We 
refer to [EFG]. 

Some strata can be described in detail. Consider the locus associated to the parti- 
tion 

t* = {g,g- 1,...,3,2}. 
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This is the penultimate stratum and it is of dimension 1. It is proved by Ekedahl and 
Oort that this locus is connected. 

Define h(g) by h(l) = 1 and h(g) = h(g - 1) x p9 p + ( ( "ff ■ Then we have 

K*{U[g,g-l,...,l]) = h (9)[T g ] 

with [T g ] given as above. 

, 3, 2} is given by 



In U$ this locus consists of a configuration of P 1 's. 

Since the degree of Ai on each copy of P 1 is p — 1 one can compute the number of 
components. This locus is highly reducible. But some loci are irreducible: 

(2.11) Theorem. For a < g the locus T a is irreducible. In particular, the locus 
Ti = Vg-i is irreducible. 

Proof. By Ekedahl-Oort (cf. [03]) we know that the locus corresponding to the diagram 
{g, g — 1, . . . , 3, 2} is connected. This implies that T a is connected for a < g. We know 
that Sing(T a ) C T a+ i, hence of codimension > 1. Actually, one can describe the normal 
bundle to T a —T a+ i. By a theorem of Hartshorne (cf. [H] ) this implies that T a — Sing(T a ) 
is connected. □ 

The irreducibility of V g -\ was also observed by Oort, cf. [03]. 
§3 Some additional results. 

We describe as an example the canonical type for hyperelliptic curves of 2-rank in 
characteristic p = 2. 

(3.1) Lemma. A hyperelliptic curve C of genus g and of 2-rank over k = k of 
characteristic p = 2 can be written as 

y 2 + y = xP(x 2 ) = x(a lX 2 + a 2 x 4 + ... + a g - X x 2g - 2 + x 29 ). (13) 

(3.2) Theorem. For a hyperelliptic curve as in (13) the canonical type is described by 
the corresponding partition fx = [g, g — 2, g — 4, ...] . In particular, the canonical filtration 
is independent of the coefficients ai. 

We refer for the proof to [EFG]. 

For g = 1 and g = 2 the supersingular locus coincides with Vq and thus occurs 
as a stratum in the Ekedahl-Oort stratification. For g > 3 this is no longer true. The 
definition of supersingular locus involves also the higher nitrations, i.e. it uses not only 
X[p], but the higher group schemes X\p l ] as well. But in a number of cases we can 
determine the class of this locus. We state here the result for the case g = 3. 
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(2.10) Theorem. The cycle class of 7r*(W M ) for \i = {g, g — 1, . . . 

x Kg) x (f[tf + (-iy)\ x A 2 A 3 . . . 



(3.3) Theorem. The class of the supersingular locus S 3 in A$ <g> ¥ p is 

[S3] = (p - 1)( P 2 - 1)( P 3 - l)(p - l)(p 2 + l)AiAa. 

The proof uses the explicit description by Li and Oort of the moduli of principally 
polarized supersingular abelian varieties, cf. [L-O]. We refer the reader to [EFG] for the 
proof and formulas for other genera. 
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